A General Bayesian Model for Testlets: Theory and Applications

The need for more realistic and richer forms of assessment in educational tests has led to the inclusion (in many tests) of polytomously scored items, multiple items based on a single stimulus (a "testlet"), and the increased use of a generalized mixture of binary and polytomous item formats. In this paper we extend earlier work (Bradlow, Wainer & Wang, 1999; Wainer, Bradlow & Du, 2000) on the modeling of testlet based response data to include the situation in which a test is composed, partially or completely, of polytomously scored items and/or testlets. The model we propose, a modi ed version of commonly employed item response models, is embedded within a fully Bayesian framework, and inferences under the model are obtained using Markov chain Monte Carlo (MCMC) techniques. We demonstrate its use within a designed series of simulations and by analyzing operational data from the North Carolina Test of Computer Skills and the Educational Testing Service's Test of Spoken English. Our empirical ndings suggest that the North Carolina Test of Computer Skills exhibits signi cant testlet e ects, indicating signi cant dependence of item scores obtained from common stimuli, whereas the Test of Spoken English does not.